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'^. • Abstract 

O I This report considers a fairly general model of constrained queuing net- 

works that allows us to represent both MMBP (Markov Modulated Bernoulli 
Processes) arrivals and time-varying service constraints . We derive a set of 
K^ i sufficient conditions for throughput optimality of scheduling policies that en- 

^^ I compass and generalize all the previously obtained results in the field. This 

leads to the definition of new classes of (non diagonal) throughput optimal 
fv^ I scheduling policies. We prove the stability of queues by extending the tradi- 

' . tional Lyapunov drift criteria methodology. 
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1 Introduction 

Networks of constrained queues have received significant attention from the re- 
search community in the last 20 years, since they provide a powerful tool for the 
C^ . analysis of complex systems, such as communication, manufacturing or transporta- 

tion networks. Specifically, in the context of computer science, networks of con- 
strained queues have been successfully applied to describe packet-level dynamics 
in wireless networks and in high speed Internet routers whose internal architecture 
is built around an Input-Queued (IQ) switch. 

Throughput and delay are the two basic performance metrics of scheduling 
policies in a generic constrained queueing or stochastic network. In their pio- 
neering work, Tassiulas and Ephremides ||T9l . have shown that optimal throughput 
performance can be achieved in networks of constrained queues by employing a 



*Dipaitimento di Elettronica, Politecnico di Torino, Italy, Phone: +39 Oil 5644076, Fax: 
+39011 564099, leonardi@mail.tlc.polito.it. 



dynamic scheduling policy according to which, at every time instant, the depar- 
ture vector maximizes the sum of the "pressures" associated with queues that are 
served. Where the pressure of a queue q is defined as the difference between its 
own weight (equal to its own length) and the weight of the queue entered by cus- 
tomers leaving q. The scheme proposed in |[T9l is referred in the literature as max 
scalar, max weight, or max pressure scheduling policy. 

Since then, a large body of work has generalized the result in |fT9l . mainly along 
four lines: i) considering more and more general models of constrained queuing 
networks; 13] [Ml ii) proposing generalizations of the max scalar scheduling policy 
that achieve optimal throughput; HI H [I5l [l6l [TTl |20l iii) looking for simple 
(low computational) heuristic scheduling policies with throughput guarantees ||2l 
m |2ll ; iv) attempting a chai^acterization of delay properties of throughput optimal 
scheduling policies |[8l[T0l[T4l[T6l[T7l. 

In particular, focusing on the second of the above mentioned aspects, works |[T] 
Il|6l[ll[lll[l5l[l6l[l7l|2a have shown that the class of throughput optimal 
scheduling policies is significantly large. It includes low complexity randomized 
scheduling algorithms mHOl, as well as, extensions of the max scalar schedul- 
ing algorithm in which queue weights are possibly non linearly related to queue 
lengths imilKlT] [l6l[T7l. Furthermore, in networks of constrained queues with 
particular symmetry properties, scheduling policies with non diagonal weight as- 
signments (i.e., when the weight of a queue may depend on the length of other 
queues) have been also shown to be throughput optimal as well ifTTlfTSl . 

Even if the collection of results already obtained in |[T]l6]l8l[m[T5l[T6l [T7ll20l . 
is rather rich, it is still far from being exhaustive. There are several obscure aspects 
that prevent full comprehension of the structure of throughput optimal policies. 
Ideally the long term final objective would be to establish a set of sufficient and 
necessary conditions for throughput optimality of scheduling policies. 

This report defines within a unique framework, a set of sufficient conditions for 
throughput optimality, which encompasses and generalizes all previously known 
results. Our analysis is based on the application of Lyapunov functions. Our 
methods, however, substantially differ from prior work because they rely on the 
application of more general Lyapunov functions, and also involve the adoption of 
some new stability criteria. For the above reasons we believe that this document 
provides a valuable contribution towai^d a deeper understanding of the structure of 
throughput optimal policies in constrained queuing networks. 

This report is organized as follows. In Sect.|2]we introduce system assumptions 
and notation. Previous work and document contribution are discussed in Sect. [3i 
Sect, previews Lyapunov drift criteria that will be invoked in the derivation of our 
main results. Sect. [5]presents our main findings on throughput optimal scheduling 
algorithms. At last we conclude the document in Sect. [6l 



2 Preliminary definitions and notations 

We consider a network composed of N physical queues (/„ with 1 < n < A^, which 
may represent, for example, either links of a wireless multi-hop network or a virtual 
output queues (VOQ) in a IQ-switch architecture. The network is traversed by a set 
T (with \T\ = F) of different customers flows, each-one characterized by a given 
ingress/egress queue in the network (sj, dj). 

We assume time to be slotted, and physical queues to have infinite storage 
capacity. Each physical queue can potentially store customers belonging to several 
flows. The set of customers belonging to flow / and enqueued in queue q^ forms 
a virtual queue Vm- The whole network can be regarded as a system of M < FN 
discrete-time virtual queues represented by row vector V, whose m-th element, 
1 < m < M corresponds to virtual queue Vm- 

The routes of customer flows in the network are fixed (a priori established and 
time invariant). Without loss of generality, we assume that all customers belonging 
to flow / and stored in queue Vm will advance to the final destination following the 
simple path along the network, which corresponds to a predetermined sequence of 
(virtual/physical) queues to be traversed. We specify network routes by means of 
an M X Af routing matrix R = [r^"^'P>] whose element r^^'^' G {0, 1} indicates 
whether customers departing from virtual queue m enter virtual queue p. Lj We 
remark that according to our assumptions since all customers of a flow residing 
in a virtual queue must reach their final destination following the same path queue 
forking is not pemiitted. Instead queue joining (i.e., multiple virtual queues feeding 
into one downstream virtual queue) is permitted. 

For any physical queue qn, function VQ{n) returns the set indices correspond- 
ing to the associated virtual queues. For every virtual queue Vm, PQ{m) returns 
the physical queue that corresponds to Vm- For any virtual queue of index m the 
function FL{ni) returns the index of the corresponding customer flow /. At last 
for every flow / FP{f) returns the ordered set of indices of virtual queues storing 
flow / customers along the associated path. 

Let Xt = {Xf. ,x\ , . . . ,x\ ) be the row vector whose m-th element xj^ , 
1 < m < M, represents the number of customers (i.e., either the number of 
packets or bits/bytes) in queue Vm at time t. The evolution of the number of queued 
customers is described by xj^{ = x^ + e^ — df^ , where ej™" represents the 

number of customers that entered virtual Vm in time interval {t,t + 1], and d^ 
represents the number of customers departed from Vm it time interval {t,t + 1]. 
Ft = {el ,e\ , . . . ,el ) is the vector of entrances in the virtual queues, and 
Dt = {d\ \dl , . . . ,dl -^ ) is the vector of departures from the virtual queues. 
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With this notation, the system evolution equation can be written as: 

Xt+i =Xt + Et-Dt (1) 

We represent service constraints among different servers of the network in the 
following general way. At every time t, the queue departure vector Dt is con- 
sti'ained to lie within a compact and convex region Vt- We remark that region Vt 
may change over time, since it is possibly controlled by a finite state discrete-time 
Markov chain at steady-state (i.e., Vt = V{S[')). Without loss of generality we 
assume V{SP) to be deterministically associated with the current Markov Chain 
state S^ . We denote by S^ the state space of Markov Chain S^ that models 
possible variable environmental conditions (such as fading conditions). Additional 
constraints, such as integrality may be imposed to departure vectors Df. However, 
we require that for every state S^ , every vertex of V{SP) represents a feasible 
departure vector (i.e., a vector that satisfies all constraints). In the particular case 
in which Pj = D (i.e. V does not vary with time) we say that the system of 
queues is subject to static service constraints. We observe that this approach is 
very general and encompasses the classical case ||T9l in which service constraints 
are represented by a contention graph. □ In the latter case V is defined as convex 
hull generated by those vectors D G {0, 1}^^ that correspond to the activation of 
all possible independent sets of nodes over the contention graph. Dt G {0, 1}*^, 
by construction, corresponds to some independent set over the contention graph, 
and therefore trivially lies in V. Our approach covers also the case in which V is 
determined by a rate-power function p{Pt, S^) that maps vectors of power allo- 
cations to seiners Pt (under some constraint on the maximum power that can be 
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employed) into vectors of service rates, for every state St , as in [141 ■ In this latter 



t 
case V{SP) is to the convex hull generated by service rate vectors corresponding 

to all possible extremal power allocations. 

The entrance vector is the sum of two terms: vector At = {al ,a\ , . . . ,al ) 
representing the customers arrived at the system from outside, and vector Jt = 
(Jt lit ■• ■ ■ ■ -iJi ) of recirculating customers; j^ is the number customers 
departed from some virtual queue and entered virtual queue m in time interval 
(t, t + 1]. Note that when customers do not traverse more that one queue (as for a 
switch in isolation), vector Jt is null for all t, and At = Et. In this case we say that 
the network is traversed by single-hop traffic. 



^ Contentions graphs are typically defined as follows: 

Definition 1 The contention graph Gi{V' ,£^) is an undirected graph in which: i) vertices v G V^ 
correspond to network (virtual) queues; ii) an edge connects two vertices v and v , if the correspond- 
ing queues can not simultaneously served. 



Let us consider the external aixival process At = (aj , a^ , . . . , a^ ) ; in 
general we suppose that the sequence At is a Markov Modulated Bernoulli Process. 
We further assume the modulating Markov Chain S^ to have a finite number of 
states. We denote by S^ its state space. At last we assume the number of amvals 
at queues to be deterministically bounded by some constant^] We denote by A = 
(A^^\ A^^^, . . . , A^^^)) the average arrival rate E[^t]. In the specific case in which 
At forms an i.i.d. sequence, we say that the traffic is i.i.d. The workload Wt 
provided by customers that in time interval [t,t + 1) entered the system of queues 
is given on average by W = K[Wt] = A{I — R)~^ , being / the identity matrix. 

Note that since Jt = DtR, the system evolution equation can thus be rewritten 
as: 

Xt+i=Xt + At-Dt{I~R) (2) 

At last, given two vector^, A € M^^ and B G M^^, we denote by {A ■ B) the 
inner (scalar) product between them {A ■ B) = AB^ = Ylim=i o.^"^'b^"^' , where 
B^ is the transpose of i?; we denote, instead, by ||A|| the Euclidean norm of A, 
\\A\\ = ^/{A^. 

In the following we will use capital letters to denote vectors and matrices, lower 
case letters to denote scalars, calligraphic characters to denote sets. Moreover we 
will denote by capital letters, functions of multiple variables while by lower case 
letters, functions of a single variable; at last, with abuse of notation, given a vector 
A, we will denote by f{A) the vector whose ?Ti-th component is f{a^"^^). 

2.1 Examples 

As first example, we consider an input queued switch with N input ports and N 
output ports. The switching fabric is assumed to be non-blocking and memoryless. 
Fixed size packets are stored at input ports. Thus one physical queue corresponds to 
every input port. Each input port maintains a separate virtual queue for each output 
port. Therefore, the switch in can be modeled as a system comprising M = A^^ 
virtual queues. Let Vm, m = iN + j be the virtual queue at input i storing packets 
directed to output j, with i,j = 0,l,2,...,N — l. 

At each time slot, the switch scheduler selects packets to be transferred from 
input ports to output ports. The set of packets to be transferred during an internal 
time slot must satisfy two constraints: i) at most one packet can be transfeixed from 
each input port, and ii) at most one packet can be transferred toward each output. 



^^We assume that of 5*^ and Sf^ evolve independently, even if tliis assumption is not strictly 
needed to obtain our results. 

■^In this report N denotes the set of non negative integers, R denotes the set of real numbers, and 
R^ denotes the set of non negative real numbers. 



Service constraints can be formalized as are: 

m&VQi{i) m&VQoU) 

where VQi{i) denotes the set of indices associated to VOQs storing packets at 
input i; and VQo{j), the set of indices of VOQs storing packets directed to output 

i- 

As second example we consider a ad-hoc network with N nodes. Every node is 
provided with a single transmitter and maintains a per destination virtual queuing 
structure. Thus, at node i packets destined to node j are enqueued in a virtual 
queue Vm with m = iM + j' . The system of queues can be modeled as a system 
of M = N^ virtual queues. Packet routes are assumed fixed; all packets at node i 
destined to node j follow the same route to their destination. 

Service constraints come from the fact that; 1) two virtual queues residing in 
the same node (i.e., insisting on the same physical queue) can not be served simul- 
taneously because of the conflict for the transmitter. 2) some pairs of virtual queues 
residing in different nodes can not be activated (served) simultaneously because of 
the mutual interference on the receivers. Service constraints are precisely specified 
by the associated contention graph G/(V^, £^). 

2.2 Stability Definitions 

Several stability criteria for constrained queuing networks have being defined in 
the technical literature: 

Definition 2 A system of queues is rate-stable if 

t-i 
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lim — = hm - > (Ei — Di] = 0. with probability 1 

i=0 



Definition 3 A system of queues is weakly stable if for every e > 0, there exists a 
& > such that: 

lim Pr{||Xt|| >6} <e 

n— >oo 

where Pr{£'} denotes the probability of event £. 
Definition 4 A system of queues is strongly stable if 

limsupE[||Xt||] < oo 

t— >oo 



Note that strong stability entails weak stability, and that weak stability entails rate- 
stability. Indeed, rate stability allows queue lengths to indefinitely grow with sub- 
linear rate, while the weak stability entails that queues are finite with probability 1 . 
This however does not guarantee that the average delay experienced by customers 
is bounded. Strong stability entails, in addition, the boundedness of average cus- 
tomer delays. 

Strong-stability concept can be generalized as follows u : 

Definition 5 Given a non-negative continuous function F{X) E 

with liniiijf ||_^oo P{^) = oo; a system of queues is F(X)-stable if 

lim supE[-F(-'^t)] < oo 



Note that F {X)-stability property becomes stricter by selecting functions F{X) 
that increase faster to oo, for large ||X||. In other words F {X)-stability entails 
G{X)-stability for any other function G{X) such that| G(X) = 0{F{X)) as|Z| 
|[X|| — > oo. In the following we will make extensive use of the F {X)-stability 
criterion. 

2.3 Capacity Region 

Given a scheduling policy vr, the stability region of a network of queues is the set 
of average aixival vectors A in coixespondence of which the system is stable (under 
one of the above criteria). Arrival vector A is said to be admissible when it lies 
in the stability region for some scheduling policy vr'. The capacity region of the 
network is the set of all average admissible aixival vectors, i.e. the set of vectors for 
which there exists some scheduling policy that makes the system of queues stable. 
With abuse of language we call admissible an arrival process whose associated rate 
is admissible. 

Under the rate stability criterion, the capacity region of the system Crate, is 
given by the set of A: 



'C'°[R -^ R] denotes the class of real valued functions that are fc-th times continuously differ- 
entiable. Furthermore given a sufficiently smooth function g(x) R — s- R we denote by g'{x) its first 
derivative, with g"{x) its second derivative, and with g^ '{x) its h-th derivative. 

^Given two functions f(n) > and g{n) > 0: f{n) = o{g{n)) means lini„_>oo f{n)/g{n) — 0; 
f{n) = 0{g{n)) means limsup„^^ f{n)/g(n) = c < oo. 

'For any function F : R+*^ ->■ R we use lim||x||^oo F{X) = I with i G R U {oo} as shorthand 
notation to mean that lim||c<||_»oo F{oiXq) = I for any Xo G R^^^ with ||Xo || = 1 



Cratn = ^A:W = A{I- R)~^ = Y, ^sDiS) \ with D{S) £ V{S),yS G S^ 

S£SO 

(3) 
where vr^ is the steady state probability associated with states S G S^ of the 
DTMC governing service constraints, and D{S) is an arbitrary vector lying in 
P(S') |[T4l [T9l . Observe that Crate is a compact (closed and bounded) set in 
^+M Under either the weak and strong stability criterion, the capacity region 
Cwoak = Cstrong coiTcsponds to the interior of Crate, i-C- to the set of average ar- 
rival vectors A, whose corresponding workloads W that can be written in the form: 
W = J2seso T^sD{S), with D{S) lying in the interior of V{S). 

3 Previous Work and Document Contribution 

In their seminal work, Tassiulas and Ephremides ||T91 have shown that under i.i.d. 
arrival processes and static service constraints, optimal throughput can be achieved 
by employing max scalar scheduling policy vTmax. according to which at every time 
slot t, the departure vector, satisfies: 

Dp^ = argmax(Xt(/ - Rf ■ D) 

DeT) 

D<Xt 

More precisely vTmax guarantees the network of queues to be weakly stable within 
the capacity region. Observe that the queue length vector Xt has to be interpreted 
as a vector of weights associated to queues, while Xt {I — R)^ is the corresponding 
vector oi pressures that takes into account the effect of customers recirculation (for 
networks of queues supporting single-hop traffic, pressures coincide with weights). 

The result in |[T9l has been extended in several respects. First, the stability 
properties of the max scalar policy have been strengthened (strong stability has 
been proved) and extended under more general non i.i.d. traffic and dynamic ser- 
vice constraints |[3l[T4l. 

Second, the class of throughput optimal schedulers has been extended, includ- 
ing max scalar policies that employ non linear queue weights. Under i.i.d. arrival 
processes and static service constraints, scheduling policies according to which the 
vector of departures satisfies: 

Df = argmax{g{X){I - Rf • D) 

Dev 
D<Xt 



where g{x) S C^ [M"^ ^> M] is a non negative function satisfying: g{0) = and 
linit^oo ^^ = 0, have been shown to be throughput optimal lITl 151 [8l [T6l [TTl [TSl . 
Particularly relevant are the cases in which g{X) = X°' for a > 0. Despite the fact 
that strong stability has been analytically proved for a < 1 veiy recently lITSl . it 
is a longstanding conjecture ||8] [161 [13 that optimal delay properties are achieved 
when a ^ 0. In |[T6l [TTl this conjecture has been supported by some analytical 
evidence. 

Non-diagonal max scalar policies achieving optimal throughput performance, 
have been have recently identified in |[T2l[T5l . In |[T5l Projective Cone Schedulers 
PCS, a new class of scheduling policies has been shown to be throughput opti- 
mal (under the rate stabihty criterion) in networks transporting single-hop traffic. 
According to PCS the departure vector at every time t satisfies: 

Df^^ = arg max(XQ • D) (4) 

D<Xt 

where Q is a positive definite symmetric matrix with null or negative out of diag- 
onal elements. Observe that according to PCS, contrarily to all previously men- 
tioned schemes, weight associated with queue v^"^' may depend on the length of 
other queues. In this case we say that the scheduling policy employs non diagonal 
weights. Moreover, we wish to mention that other examples of policies employing 
non diagonal weights have been earlier shown to achieve throughput optimality in 
constrained queuing networks with particular structures, such as those correspond- 
ing to IQ switches (see for example LPF for IQ switches 111 [HI). 

A different, fairly general result has been obtained in |[T2| . For a general net- 
work with static service constraints, given a function G{X), G ^ C^ [R"*"^ — )• R+], 
the scheduling policy: 

^VGmax ^ argmax(VGo(^t) • D{I - R)). (5) 

Dec 
D<Xt 

has been proven to be throughput optimal, provided that \/G{Xt) is Lipschitz con- 
tinuous, G{Xt) is monotonic; i.e. VG{Xt) G ]R+^^ for any Xt G M+^^ and 
||VG(Xt)|| ^^ oo as \\Xt\\ — ?• cxo. Observe, however, that the requirement for 
G{X) to be monotonic in the previously specified sense, severely reduce the do- 
main of applicability of the result in |[T2| . For example, non trivial Projective 
Cone Scheduler (with negative out of diagonal elements) are not monotonic. Our 
analysis generalizes flT] making a further significant step in the direction of the 
identification of the most general conditions for G{X) that guarantee throughput 
optimahty of the associated max-scalar pohcy. 



Scheduling policies with memory ||6] [13] |20l represent a further example of 
throughput optimal schemes for networks with static service constraints. The 
schemes proposed in ||6l [13] |20l are based on the idea of generating an admissible 
candidate departure vector D'l at every slot, according to some simple rule; then 
the departure vector D^^m j^ selected between D^ and D^^ by maximizing the 
associated aggregate pressure Z)™'^™ = argmax{(X • Df), {X ■ D^f^)}. It has 
been shown that such schemes achieve optimal throughput (i.e., strong stability) 
under admissible i.i.d. amval processes and static constraint conditions, provided 
that at every slot it can be guaranteed D^ = arg max(X • D) with a probability non 
less than J > 0. Notice that the above condition is satisfied when D^ is uniformly 
selected among the vertices of V. 

This document provides several contributions with respect to previous work: i) 
Theorem |5]and|6]significantly extend of the class of thr^oughput optimal max scalar 
like policies policies exploiting non linear and non diagonal weights. In particular 
with respect to f\T\ , Theorems [5] and [6] do not require G{Xt) to be monotonic. 
Furthemiore throughput optimality is proven under a general model of constrained 
queuing networks possibly subject to dynamic service constraints and non i.i.d. ar- 
rivals, ii) Theorems |7] and [8] generalize the class of throughput optimal scheduling 
algorithms with memory, applying, for the first time to the best of our knowledge, 
the concept of schedulers with memory to network of constrained queues subject to 
dynamic service constraints, iii) Furthermore we are able to strengthen the above 
results, showing that every polynomial moment of the queue lengths remains finite 
under any of the above schemes, as long as the average arrival vector lies within 
the capacity region, iv) At last, from a methodological point of view, we introduce 
new Foster-Lyapunov drift conditions for F(X) -stability (reported in Sect. @]i, ex- 
tending in such a way previous drift arguments. 

4 IVTarkov State and Lyapunov Stability Criteria 

Under previous assumptions, the process describing the evolution of the system of 
queues is an irreducible Discrete-Time Markov Chain (DTMC), whose state vector 
at time t, Yt = {Xt, St), is the combination of vector Xt and a vector St that 
represents the memory of the system in the case in which arrivals are not i.i.d. 
and/or service constraints are dynamic. 

Let T-L be the state space of the DTMC, obtained as Cartesian product of the 
state space |j <%" C N^^ induced by the queue lengths vector Xt and the state space 

5 = S^ X S^ C N"^ induced by St, we further assume S to be a finite state space. 
Note that V. c n+^ with H = M + S. 



] denotes the set of non negative integers. 
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From Definition m we can immediately see that DTMC Yt is positive recurrent, 
if and only if the system of queues is weakly stable (we recall that the DTMC 
modelling the system is assumed to be iiTeducible). 

The following general criterion for the (weak) stability of systems is therefore 
useful in the design of scheduling algorithms. This theorem is a straightforward 
extension of Foster's Criterion; see 



Theorem 1 Given a system of queues described by a DTMC with state vector Yt = 
{Xt, St) G N^, whose state space % is the Cartesian product of the denumerable 
state space X C N (with Xt € X), and a finite state space S € N (with 
St G S); if a lower bounded continuous function CiY), called Lyapunov function, 
C : M"*" — > M can be found such that: 

E[CiYt+i)\Yt]<C{Yt) + vo (6) 

for some vq < oo, and 

E[C{Yt+i) - C{Yt) I Yt] < -e VF* : \\Xt\\ > b, (7) 

for some e € M^ and b S M^; then the DTMC is positive recurrent and the system 
of queues is weakly stable. 

Remark: observe that for every 1^ : ||Xi|| > fe, the satisfaction of© immediately 
follows from (jT]) (with vq = 0). Therefore, it is sufficient to verify (|6) for Yt : 
||Xj|| < b and © to apply the above Theorem. The following result provides a 
criterion for strong stability. 

Theorem 2 Under the same assumptions ofTheorem\I} if CiY), additionally sat- 
isfies: 

E[/:(ym) - ^{Yt) I Yt] < -e\\Xt\\ yYt : \\Xt\\ > b, (8) 

for some e € M"^ and b € M'^; then the system of queues is strongly -stable. 

Previous stability criteria can be also applied to establish the stability of a DTMC 
Yt^, obtained by sampling Yt in correspondence of an opportunely defined se- 
quence of time instants. In particular we are interested in the case in which t^ G N+ 
foiTii a sequence of stopping times: 

Definition 6 A sequence of random time instants tj^ € N"*" is a sequence of non- 
defective regeneration instants (or stopping times) for the evolution of a system 
of queues if for any k, the occurrence or non-occurrence of the event {t^ = t} 
depends only on the values ofYi,Y2,Y3---Yt. Moreover, letting Zk = tfc+i ~ tk> 
E[z^] < oo, for any h G N+. 

11 



From the strong Markov property 11221 immediately follows that the evolution of 
Markov Chain Yt after t^ is conditionally independent of the evolution of the sys- 
tem before tk, given the state Y{tk), provided that tk is a stopping time. We remark, 
instead, that the above conditional independence property does not hold if t^ is a 
generic random time. 

In this case, from the strong stability of Yt^. it is possible to infer strong stability 
of the original system: 

Theorem 3 Under the same assumptions of Theorem |7] and the additional as- 
sumption that both arrival vectors, At, and departure vectors, Df, are bounded in 
norm, if a lower bounded continuous Lyapunov function CiY), V : M+^ — ?> M 
can be found such that for some vq < oo, for an opportunely defined non-defective 
sequence of regeneration instants {tk}: 

nC-{Yt,^,)\Yt,]<C{Y,) + v, (9) 

for some vq < oo, and 

E[>c(yi,^j - c{Yt^) I y^j < -e\\xtj yYt^ ■. \\Xtj > b (lo) 

for some e € M^ and b € M'^; then the system of queues is sivon^y -stable. 

A brief proof of this statement is in Appendix |A] 

Lyapunov drift arguments can be extended to obtain the following criterion for 
F(X)-stabiUty: 

Theorem 4 Under the same assumptions of Theorem \1\ if a lower bounded con- 
tinuous Lyapunov function CiY), C : M"'"^ -^ R can be found satisfying the 
following two conditions: 

E[C{Yt+i)\Yt]<C(Yt) + vo (11) 

for some vq < oo, and 

E[C{Yt+i) - C{Yt) I Yt] < -eF{Xt) Vy* : ||Xt|| > b (12) 

for some e G M"^, b G M"^, and for a function F{X) : M -^ M non-negative, 
continuous with lini|jx||-5>oo P{^) = oo,' then the system of queues is F{X)-stable. 

The proof is reported in appendix. 

At last, using similar' ai'guments as in Theorem |3] we can easily derive the 
following result: 
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Corollary 1 Under the same assumptions of Theorem [7] and the additional as- 
sumption that that both arrival vectors At and departure vectors Dt are bounded 
in norm, if a lower bounded continuous Lyapunov function C{Y), L : M"*" — )■ M 
can be found such that, for an opportunely defined sequence {t^} o/ non-defective 
regeneration instants and for some Vq < 00/ 

E[C{Yt^_,,)\Yt,]<C{Yt) + vo, (13) 

and for some e S M^ and b € M"*"; 

miYt,^^) - C{Yt^) I Yt,] < -eF{Xt^) ^Yt, : \\XtJ > b (14) 

being F{X) : M ^- M, a non-negative, continuous function satisfying 
lini||x||->>oo F'{X) = 00; then the system of queues is F(X)-stable. 

5 Main Results 

In this section we introduce the class of scheduling policies that achieve optimal 
throughput performance. To improve the readability of the section, all proofs have 
been moved to Appendix IbI 

Definition? Given any function G{X), G G C^[M+^ -> M], we define as 
VG(X)-max scalar, the scheduling policy vrvGmax that selects the departure vec- 
tor according to: 

^VGmax ^ argmax(VG(Xt)(/ - Rf ■ D), (15) 

D<Xt 

i.e., the vector of departing customers D^^^^^ € T^{3t) i^ chosen so to maximize 
the inner product between the departure vector itself, and the gradient of G{X) 
evaluated at Xt, ('VG(X) \x=Xt' denoted for short by VG{Xt)}, multiplied by 
the transpose of matrix [I — R). 

Note that VG{Xt){I — R)'^ can be interpreted as the vector of pressures associated 
with the weight vector VG{Xt). Furthermore, observe that since {'VG{Xt){I — 
R)T . D) = {VG(Xt) ■ D{I - R)), the VG{X)-max scalar can be defined as well 
as scheduling policy according to which: 

^VGmax ^ argmax(VG(Xi) • D{I - R)). (16) 

Dec(St) 
D<Xt 
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At last, in the relevant case in which the network is traversed by single-hop traffic, 
i.e. when R = 0, D^Gmax satisfies: 



D7^"^^ = argmax(VG(Xj )•£)). (17) 

De-D(St) 

D<Xt 

The following two theorems provide conditions for throughput optimality of 
VG{X)-max scalar scheduling policies. We denote with Hg{X) the Hessian of 
G() evaluated at X. We recall that an arrival processes is said admissible if its 
associated average workload W = A(/ — R)^^ lyes in the convex hull of the of 
the feasible departure vectors, i.e., departure vectors that satisfy service constraints. 
We denote with Hg{X) the Hessian of G at A 

Theorems The network of queues is \VG{X)\-stable under i.i.d. admissible 
arrival processes and static service constraints, whenever a VG(X)-max scalar 
scheduling policy is employed, provided that G(X) is in C^ [M+'^ -^ M] and satis- 
fies the following technical conditions: 

1. G{X) grows to infinity faster than \\X\\ when X grows to infinity, |i|/.e..' 

lim -4-f = oo: (18) 

||X|i^oo \\X\\ 

2. G{X) exhibits a sub-exponential behavior for large X; i.e, 

G{X + Y) ^ ^. {VG{X + Y)-Z) ^ ,. ZHGiX + Y)Z 



T 



lim — „, ,„ — = 1, lim — , ,„ — — — = 1, lim 



G{X) ' iixiKoo (VG(X)-Z) ' ijxiHoo ZHg{X)ZT 

(19) 
for arbitrary vectors Y, Z; 

3. the following conditions on the orientation of\/G{X) are met: 

{VG{X){I -Rf -D) <0 VZ) > s.t. {X-D) = 0. (20) 

and 

{VG{X){I-RY -D) 

IliSoo ||VG(A)|i > 'f^'-^'' ^ ^ ' ^''^ 

Stability properties of VG{X)-max scalar scheduling policies can be extended 
to more general Markov Modulated Bemoulli Process (MMBP) arrival processes 
and dynamic service constants, when G{X) satisfies slightly less general condi- 
tions: 



'We recall that for any function F : R+" -> R we use lim||x||^oo F{X) = I with I G RU {cx)} 
as shorthand notation to mean that lim||a||^oo F{aXo) = / for any Xo £ R+^^ with \\Xq\\ = 1 
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Theorem 6 The network of queues is \\7G{X)\-stable under admissible MMBP 
arrival processes and general service constraints whenever a VG(X)-max scalar 
scheduling policy is employed, provided that G{X) is in C°°[R"'" — )• M] and 
meets the following two conditions: 



limsup ||(9^"G)(X)|| < oo for some Hq G N; (22) 

||X|j^oo 



lim 

IXII— !>oo 



(a^+iG)(X) 



(9^)G(X) 



V/i< /lo; (23) 



in addition to lilSi . f l20D and (|27 



Observe that conditions (|22] | and (l23T i. which entail ([19] ). express the fact that the 
dominant behavior of G{X) for ||X|| — > oo is polynomial. 

When G{X) satisfies the technical conditions specified by Theorem |5] we say 
that it is a weak-potential for the system of queues; we, instead, say that it is a 
strong-potential for the system of queues, when G{X) satisfies the additional tech- 
nical conditions specified by Theorem |6] We recall that the proofs of Theorems |5] 
and |6] are in Appendix IB] 

Note that according to Theorems |5] and |6] |VG(X)| -stability has been proved 
for VG{X)-max scalar policies in non overloaded conditions. |VG(X)|-stability 
may become weak, especially when \/G{X) increases slowly to infinity for 
\\X\\ -^ oo. For example if G(X) = j^ I]m(a;^™^)^+" for a < 1 (i.e. 
VG{X) = X°'), strong stability of the network of queues is not guaranteed by 
the above mentioned Theorems. The following Corollary allows to strengthen the 
assertions of Theorem|5]and Theorem|6] showing that V G{X)-max scalar policies 
associated with weak/strong potentials guarantee that every polynomial moment of 
queue-lengths remains finite within the capacity region: 

Corollary 2 Consider a weak potential function G{X); the network of queues is 
\\X\\^-stable, for any h £ N (i.e., every polynomial moment of the queue lengths 
is finite), under admissible i.i.d. arrival processes and static service constraints, 
provided that the associated VG(X)-max scalar scheduling policy is employed. 
When, instead, G{X) is a strong potential function, \\X\\ -stability can be proved 
for any /i € N, under MMBP arrival processes and dynamic service constraints. 

Again, we recall that the proof of the Corollary is in Appendix IbI 

Remark: The class of scheduling policies that satisfy the assumptions of The- 
orem|5](or Theorem |6} is fairly lai^ge and comprises the following three subclasses 
of optimal policies, as particular cases. Indeed note that: 
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1. Any function G{X) in the form: G{X) = ^^ = g{x^"^^) where g{x) a 
function in C^ [R+ — > M] with a super-linear and sub-exponential asymptotic 



behavior, (i.e. g{x) such that: lima;^oo ^ = 00, lim^^oo ^j^ = !> and 
lim2:_s.oo ^Tjy = ^ ,(^) = 0), and with the first derivative null in the origin 
ig'{0) = 0) is a weak potential. Furthermore If g{x) is in C°°[M+ — )• R] and 
has a polynomial asymptotic behavior for lai^ge x, (i.e., lim sup^^^^ (^(^0) < 

CX3 for some ho G N, and lim2,._5.oo ^ (fe),- f = V/i < ho), then G(X) 
is a strong potential. The associated VG{X)-max scalar policy, according 
to which D = argmax(/i(X) • D) with h{x) = g'{x) achieves ||X||'*- 
stability for any h. With abuse of language when g{X) satisfies the above 
conditions, we say that it is a weak (strong) scalar potential. For this subclass 
of scheduling policy, we extend findings in |[l][8][16j[T7J[T8l, since we prove a 
stronger form of stability (the finiteness of every polynomial moment) under 
a more general network model with possibly coixelated aixivals and dynamic 
service constraints. As a particular case, if we select f{x) = -^1:^: we obtain 
Dt = aTgm.ax{X°'-D). By choosing, instead /(x) = (3;+l)(log(3;+l) — 1) 
we can prove stability properties of the scheduling policy according to which 
Dt = argmax(log(l + X) ■ D). 

2. Choosing G{X) = {g{X)Q ■ g{X)) we obtain another subclass of func- 
tions satisfying the assumptions of Theorem [6] for networks transporting 
single-hop traffic, provided that Q is a positive definite symmetric matrix 
with non positive out-of diagonal elements, and g{x) is C°°[M"'' -^ M], 
increasing, null in the origin (i.e., g{0) = 0) with polynomial asymptotic 
behavior for large x, (i.e., lim.suTp,j.^^ g^^°^ < 00 for some ho G N, and 

lim^^oo ^g(h)(^^^ = V/i< ho) and such that liniaj^oo ^ = 00. 

This class of functions extends the class of scheduling policies proposed 
in lITSl . for which D = argmax(XQ • D) (obtained when g{x) = x). 
Once again, we wish to emphasize this class G{X)-max scalar policies is 
not covered by |[T2l . since G{X) is not monotonic, as effect of the negative 
out of diagonal elements of Q. 

3. For networks transporting single-hop traffic, every G{X) in the foim 
G{X) = f{X)PX can be easily shown to be a strong potential, provided 
that: i) P is a positively defined symmetric matrix with strictly positive di- 
agonal elements pmm > 0, ii) f{x) is given by: 

y/^N = I X- 2e'T^ - 2e-^ for x < 1 
1 X — 2e~^ for X > 1 
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In particular, the above function satisfies (|20] | since /(O) = and 
f'{x) \x=o< 0. This class of policies generalizes the LPF policy IT] [TTl 
defined for input queued switch architectures. To establish a clearer re- 
lationship between LPF and the class S/G{X)-max scalar policies with 
G{X) = f{X)PX, we focus on single-hop networks of queues with static 
service constraints. Without loss of generality, we assume service constraints 
among virtual queues to be represented by a contention graph. For any vir- 
tual queue Vm we can define X^, the set of virtual queues that are conflicting 
with Vm- We conventionally assume Vm S Im- Then taking matrix P, such 
that; its element Pm,m' = I H rn' G Im (and by construction m G I'^) 
and Pm,m' = otherwise; we obtain a max scalar scheduling policy whose 
associated queue weights satisfy: 

when Vm is not empty. Weight Wm is instead negative, when Vm is empty. 

Now if we consider an IQ switch architecture queue architecture, for any 
VOQ Vm, 1m is, by construction, composed of all the virtual queues resid- 
ing on the same input port or directed to the same output port of the VOQ 
Vm- Thus, VG{X)-max scalar scheduling policy associated to G{X) = 
f{X)PX degenerates into a slightly modified version version of the LPF 
policy. The difference between the weights of classical LPF and the VG{X)- 
max scalar policy for G{X) = f{X)PX, is represented by coixection terms 
e~^lx^>o- At last, observe that G{X) is not monotonic, since weights as- 
sociated to empty queues are negative. 

The above three sub-classes of optimal policies are not at all exhaustive. For ex- 
ample, functions in the form G{X) = {g{X)P ■ g{X)) can be easily proved to 
be strong potential functions for general constrained networks, provided that P 
is a symmetric positive defined matrix with strictly positive diagonal elements 
Pmm > 0, and g{x) is C°°[M~'^ -^ M], increasing, null in the origin (i.e., 
g{0) = 0), with null derivative in the origin (i.e., g'{0) = 0), polynomial asymp- 
totic behavior for large x (i.e., lira sup ^^ ^ g^^'''^'' < oo for some /iq G N, and 

lim^_>.oo ^ (h)( ) = V/i < /iq), and such that lim^.^oo ^^ = c«. Particularly 
relevant are functions in the form G{X) = (X"+^P • X°+i) with a > 0. 

The following result allows to more precisely characterize the class of well 
defined potential functions: 

Corollary 3 Given a weak (strong) monotonic potential function Gi{X) f/.e., a 
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potential function with VG'i(X) > Q for any X >Q) and a weak (strong) potential 
function G2{X), then also 

• G{X) = aGi{X) + PG2{X) Va,/3>0 

. G{X) = Gi{X)G2{X) 

are weak (strong) potential functions. 

Furthermore given G{X), weak (strong) potential function and g{x) € 
C2[M+ -^ M], (g{x) e C°°\U+ -^ W\), increasing with at least lin- 
ear and sub-exponential (polynomial) asymptotic behavior, i.e. such that: 

}\m inf S^ ■> n lim g(^+^) - 1 lim sISeI - n \\m g"(^) - n 

for limsup^^oo fi'^'^^H^) < °o, for some Iiq and lima;_^oo ^ (h)/ V = 0, V/i < 
/iqJ, f/je« g{G{X)) is a weak (strong) potential function. If additionally g'{0) = 
also G{g{X)) is a weak (strong) potential function. 

The proof, which consists in the verification that all conditions of the statement 
of Theorem |5] (Theorem |6]l are met, is rather long and tedious even if conceptually 
sti'aightforward. For these reasons, we omit it. 

Previous corollary characterizes the algebraic structure of potentials and makes 
the verification of throughput optimality easier for \7G{X)-max scalar policies as- 
sociated with potentials with complex structure such as: G{X) = J2mS{x ) + 
(Xi+°P . Xi+"), G{X) = E^5(2;(™)) • {X^+'^P ■ Xi+") or G{X) = 
g{{X^^'^ P ■ X^'^'^)), where 5f(x) is a scalar potential and Pis a symmetric positive 
defined matrix with strictly positive diagonal elements Pmm > 0. 

The following Corollary allows us to further extend the class of throughput 
optimal scheduling policies: 

Corollary 4 Given a weak (strong) potential function G{X), any scheduling pol- 
icy VTvGimp achieves the same throughput performance (queue stability in non 
overloaded conditions) of the associated vrvGrnax policy, if it satisfies the following 
property: 

lim ((D^Gmax _ ^VGimp) . VG(X)(/ - Pf) = o(|lG(X)|l). (24) 

||X||— !>oo 

The proof is reported in Appendix |B] 

In general, it is easy to see that scheduling policies according to which: 

^VGmax ^ argmax(VG(Zt)(/ - Pf ■ D) (25) 

De-D(St) 

D<Xt 
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meet constraint (|24| | as long as E[(||^j — -'^tH)] is bounded for any /i G N. 
Thus, the class of throughput optimal scheduling policies includes VG{X)-max 
scalar policies operating with imperfect/delayed queue status information as well 
as frame-based VG{X)-max scalar policies (i.e., policies in which the computa- 
tion of a new departure vector is not executed at every slot, but just once a while), 
etc. 

5.1 Policies with memory 

A further extension to the class of throughput optimal policies can be provided, 
considering scheduling policies with memory ||6] [13] |20] : 

Theorem 7 Given a weak potential function for the system of queues G{X), sat- 
isfying: 

IlilPoo \\VG{X)\\ = ' ^''^ 

for some /3 > 1. The network of queues is \\X\\ -stable for any /i G N under i.i.d. 
admissible arrival processes and static service constraints whenever a scheduling 
policy with memory vrvGmcm i^ employed, provided that: 

1. departure vectors selected by vrvGmcm satisfy the following monotonicity 
property: 

{VG{Xt+i){I-Rf-Dff,^'n > {VGiXt+iXl-Rf-Df'^^'n (27) 

at every t; 

2. with a probability no smaller then 5 > 0, the selected departure vector 
j^VGmem satisfies: 

jjVGmcm ^ arg max(VG(Xt) • D{I - R)), 
Dev 
D<Xt 

i.e. Df = /)^G'™ax ^jY/j probability at least 5, at every t. 

The proof is reported in Appendix |B] 

As already mentioned it is possible to simply implement a scheduling policy 
satisfying Theorem |7] requirements by generating at random an admissible candi- 
date departure vector Df , and selecting the departure vector /^^G'^cm according to 
the rule D^Gmem ^ argmax{(X • Df), {X ■ 1)^^^'^'='^)}). 
Remark: Obser^ve that in this case the space state of the DTMC representing the 
evolution of the system of queues must be properly defined, by representing also 
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the information about the last employed departure vector. A natural choice is to 
take Yt = [Xt.Df^] with Df' = ^VGrncm purther notice that (|26]i is satisfied 
whenever G{X) exhibits a polynomial behavior for large \\X\\. 

When G{X) is a strong potential, previous result can be extended under more 
general assumptions on arrival processes and service constraints. In this latter case 
however the complexity of the scheme significantly increases, since the scheduling 
policy has to memorize the last selected departure vector for every possible state 
5 € S^ of the Markov Chain representing sei-vice constraints evolution. 

Theorem 8 Let G{X) be a strong potential function of the system of queues. The 
network of queues is \\X\\^ -stable for any /i G N under admissible MMBP ar- 
rival processes and general service constraints, whenever a scheduling policy with 
memory vrvGmcm is employed, provided that: 

1. at every time slot t, the following property is satisfied by departure vectors 
selected by vrvGmcm-' 

{VG{Xt+,){I-Rf-DYfrn > {VG{Xt+^){I-Rf-Df^{S^^,)), 

(28) 
where D^^^S^i) is the departure vector employed by the scheduler 
TrvGmcm cit the last epoch t^ < t + 1 in which S[^ = Sj^i; 

2. for some 6 > 0, the selected departure vector jji^Gxacm ^^fj^jigs- 

^VGmcm ^ arg max(VG(Xt) • D{I - R)), 

DeT>{St) 

D<Xt 
i.e. Jji^Gracrci _ jjlWGvciiai with probability non smaller than 5, at every t. 



The proof is reported in Appendix |B] Observe that the property expressed by 
represents the natural extension of (l27i to the case dynamic constrains scenario. 
To satisfy such property, the algorithm has to memorize the last selected departure 
vector D^^^'^^iS), for every possible state S G S^. Indeed (|27] ) can be obtained 
by comparing, at time t + 1 the candidate departure vector D^^^ and memorized 
vector i?f(Si^i)- 

6 Conclusions 

The research on throughput optimal scheduling policies in constrained queuing 
networks has mainly focused on the analysis of max scalar scheduling policies 
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employing diagonal weights. Only recently |[T2l[T5l . the existence of a class of 
throughput optimal max scalar policies employing non diagonal weights has been 
proved for arbitrary networks. In this document, we have derived a general set 
of sufficient conditions for throughput optimality that lead to significant extension 
of results in |[T2l[T5l . defining a large body of non diagonal throughput optimal 
scheduling policies. Furthermore, we have shown, how low complexity schedul- 
ing policies with memory can achieve optimal throughput properties under gen- 
eral conditions (i.e., under non i.i.d. arrival processes and dynamic services con- 
straints). This document contributes to make a step toward full comprehension of 
the structure of throughput optimal scheduling policies in constrained queuing sys- 
tems. The analysis of delay properties for non diagonal scheduling algorithms is 
still an important challenging open issue. 
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A Proofs of Theorems in Section 4 

Proof of Theorem |3j 

The fact that DTMC Yt^. is strongly stable, i.e., limsup^._^oQE[||^jfc ||] < oo 
is an immediate consequence of © and (ITOl ) 1221 . Then, considering a generic 
instant t and denoting by T{t) = maxjtfc < t}, we have: 



n\\xt\\]<n\XTit)\\]+E 



i-1 

Y, \\A,-D,{I-R)\\ 

-=T{t) 



Where E[|| Et=nt) ^- " ^-(^ " ^^H] ^ KEt=r(t) Ur - D,{I - R)\\] < 
E[i — T{t)]c where c is an upper bound for At — Dt{I — R) (which are bounded 
by assumption). The assertion follows letting t — > oo. Indeed limsupt_j.oo E[i — 
T{t)] < oo as consequence of standard renewal arguments. Indeed we recall that tk 
is, by assumption, a sequence of non defective regeneration instants (i.e. Kiz^.] = 
E[{tk+i - tkf] < oo). 

Proof of Theorem IH 

Since the assumptions of Theorem [1] are satisfied, every state of the DTMC is 
positive recunent and the DTMC is weakly stable. In addition, to prove that the 
system is F(X)-stable, we shall show that limj_^oo supE[-^(-'ft)] < o°- 

Let Tib be the set of values taken by Yn for which \\Xt\\ < b (where (IT2l ) does 
not apply). It is immediate that T-i^ is a compact set. Outside this compact set. 
Equation (IT2l ) holds, i.e. 

E[C{Yt+i) - C{Yt) I Yt] < -eF{Xt) ^Yt0ib 

Considering all y^'s that do not belong to T-Lb, we obtain 

¥\C-{Yt+i) - C{Yt) I YtiUb] < -eE[F{Xt) \ Yt0ib] 

Instead, for Yt € Hb, since Tib is a compact set and C{Y) continuous we have: 

sup E[C{Yt+i) I Yt] < max C{Yt) + vq < oo. 
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Denoting by c = maxy^g-^j^ ^O^t) + vq and combining the two previous expres- 
sions, we obtain 

E[£(yt+i)] < cPT{Yt G Tib} + MYt^nt} ■ {E[C{Yt) I Yt(^nb\ -eE[F{Xt) \ Ft^^]} < 

<c + E[C{Yt)]-eE[F{Xt)] + co 

where cq is a constant such that cq > {-E[C(Yt) \ Yt G T-Lb]+ eE[F{Xt) | Yt G 
"HbJlPrlyj G "Hfo}. Note that cq can be chosen finite, being Tib a compact set, and 
both F{X) and C{Y) continuous. 

By summing over all t from to tq — 1, we obtain 

TO ~ 1 

E[C(Yr,)] < Toe + E[CiYo)] - e ^ E[F(Xt)] + tqCq 

t=0 

Thus, for any tq, we can write 

TQ-l 



e 



1 1 

J^ E[F(XO] <c+- E[C{Yo)] - - E[jC{Y^,)] + cq 



To ^ To To 

E[>C(yT-o)] is lower bounded by definition; assume E[/^(yro)] > ci. Hence 

TO - 1 -. 

- V E[FiXt)] <c+- E[jC{Yo)] - - + CO 

7"0 ^ To To 

For To -^ oo, being E[>C(yo)] and ci finite, we can write 

TO-l 

limsup — V E[F(Xt)] < c + Co 

To^oo TO ^^p 

The assertion immediately follows. 



B Proofs of Theorems in Section 4 

Before proceeding with the proofs of the Theorems in Section |5] we recall some 
standard consequences of Taylor Theorem, of which we will be make extensive 
use, and we prove three useful Lemmas: 

Proposition 1 Let G{X) G : [M+^ -^ M] be h-times continuously differentiable 
over an open ball B centered at a vector X. Then, for any Y such that X + Y G B, 

h-l 

G{X + Y) = Y, -Y\d'G){X) + i?g')(X,y) (29) 



^=,' 
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where the h-order remainder w^ {X,Y) is given by: Rq {X,Y) 



]^Y^{d^G){X + pY), for some p = [0,1]. 

In particular if G{X) is twice continuously differentiable over an open ball B 
centered at a vector X, recalling that VG{X) denotes the gradient of G at X, 
and Hg{X) denotes the Hessian of the function G at X, for any Y such that 
X + y G ;B, we have: 

G{X + Y) = G{X) + d^} (X, Y) 

with R^q\x, Y) = {VG{X + /3y) • Y) for some ^ G [0, 1], and: 

G{X + Y) = G{X) + ( VG(X) • Y) + R^ {X, Y) (30) 

r'§\x,Y) = IyHg{X + /3y)y^ for some /5 G [0, 1]. The above Taylor ex- 
pansion can be generalized to vectorial functions applying ( [29l ) component-wise. 
In particular we will make use of the following result. Given G{X) twice continu- 
ously differentiable over an open ball B centered at a vector X for any Y such that 
X + Y £B,and any Z e M^ we have: 

{VG{X + Y)-Z) = {VG{X) ■ Z) + <i(X, y, Z) (31) 

with i?^^J.(X, y, Z) = (V((VG(X + PY) ■ Z)) ■ Y) = \ZHg{X + PY)Y^ for 
some /5 G [0, 1]. 

Lemma 1 IfG{X) satisfies the conditions ofTheorem\5\ then: 

lim {VG{X) • X) = oo 

||X||^oo 

X being the normalized vector parallel to X 

Proof: The proof can be immediately obtained by applying THopital's rule to 
the indefinite form (|T8] ): 

a— >oo \\X\\ a— >oo q; 

and recalling that limQ_^oo(VG(QX) • X) = \m\\\x\\^oo^ G{X) ■ X) exists in 
light of (IT9l). Observe as immediate consequence of previous statement we get: 

lim ||VG(X)||=oo 

||X||-)-CxD 
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Lemma 2 IfG{X) satisfies the conditions of Theorem\5\then: 

G{X + Y)-G{X) = R('\X,Y) = !. ^^H^x™ as\\X\\^oo, 

(32) 
whenever Y is an arbitrary vector. IfG{X) satisfies the conditions of Theorem^ 

^[G{X+Y)]-G{X) = E[i?g^(X,y)] = I ^(^^J^^l^^"^ as \\X\\ ^ oo, 

(33) 
whenever Y is a random vector with finite polynomial moments E[| |5^| | ] < oo V/i. 
Similarly, ifG{X) satisfies the conditions of Theorem^ 

{VG{X + Y),Z) - {VG{X),Z) = R^l{X,Y,Z) = | ^(||'^(^^)|[}^ as \\X\\ ^ 

(34) 

whenever Z and Y are two arbitrary vectors. If G{X) satisfies the conditions of 
Theorem^ 

E(VG(X + y),Z) - (VG(X),Z) =E[<g(X,y,Z)] = I ^(||'^(^^)|[}^ as \\X 



(35) 



whenever Z is an arbitrary vector and Y is a random vector with finite polynomial 

moments W\\\Y\\^] < ooj V/i. 

At last, ifG{X) satisfies the conditions of Theorem^ 

R^'\X,Y) = 0{\\Hg{X)\\) R^'Hx,Y) = o(||VG(X)||) (36) 

for any vector Y. IfG{X) satisfies the conditions of Theorem^ 

E[R^^\x,Y)] = 0{\\Hg{X)\\) E[R^^Hx,Y)]=o{\\VG{X)\\) (37) 

whenever Y is a random vector with finite polynomial moments E[| 1^ 1 1^] < oo V/i. 

Proof- 
Properties (|32] i and (|34l ) are an immediate consequence of the sub-exponential 
behavior of G{X), i.e (|T9l ). Now focusing on (1331 ). observe that expanding G{X) 
in Taylor series around X, we obtain: 

ho-l -. 

E[G{X + Y)] = GiX)+E[J2 ^Y\d'G){X)]+]E[R'S°\x,Y)] 
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where E[r'g''\x,Y) = ^^[Y^h(^d^Q-^^x + f3Y)] < 

^E[\\Y^^\\]supzm+'^\\i(^^G){Z)\\ < oo, because by assumptions E[Yf^] 
is bounded as well as sup^gjg+jv ||((9'^G)(Z)|| < oo (recalling (|22]|). Thus 
the last term is negligible with respect to G{X) and ||VG(X)|| since both 
G{X) — )• oo (by hypothesis) and ||VG(X)|| -^ oo (by Lemma[l]l as ||X|| -^ oo 
b). Furthermore, EE.f=rH^'(^'^)(^)] = Htl^ 1^[YW'G){X) = 
0(||VG(X)||) = o{G{X)), since E[Y'] < oo and \\{d'G){X)\\ = o{d'^^Glx)) 
for any I < i < ho, from (|23] ). Thus (1331 ) is proved. (l35T l can be proved repeating 
the same arguments to every component of VG{X). 

(l36l ) can be proved observing that by definition R\ylj{X,Y,Y) and 
rP{X,Y) are closely related, indeed: R^^l.{X,Y,Y) = YHg{X + 

l3iY)Y^ for a /3i G [0.1], while i?^J.(X,y) = y//G(^ + p2Y)Y'^ 
for a /32 G [0-1], possibly different from /3i. Now by (IT9l ) we get 

that limii^ii^^ 4l(W) " l^ll^ll^- SSwf^ = ^' ^"'*^'" 

more from (|34l ) we have lim||xj|^oo fvcfx)!! ~ *-*' '^^'^ ^^ alterna- 

tive lmimf||x||^oo Jhg{X)\ \ > ^^^ limsup||j^||_,^ Jhg{X)\\ > °°) 

thus combining both we get: lim||;^||^oo \\^g(x)\ ~ ^ ^^^ ^^ ahemative 

liminf||x|j-^oo \\'hg{X)\\ > ^ ^^'^ ^^t^s^P\\x\\~,oo \\Hg{X)\\ > °°) 

At last ^ can be proved observing that: E[G{X + Y)] = G{X) + {VG{X) ■ 

E[Y])+E[RS\x,Y)] = G{X) + {VG{X)-E[Y])+E[Et2^Y\d'G){X)] + 

E[R'^°\x,Y)]thus: 

ho-l .. 

E[r'§\x,Y)] = E[Y^ -Y\d'G)iX)]+E[Rp'\x,Y)] 

i=2 * 
Now from (|22] ) and (|23] ). as before, we can conclude that all terms on the right are 

0(||FGG(X)||)ando(||VG(X)||). 

Lemma 3 IfG{X) satisfies the conditions ofTheorem\5\(and in particular condi- 
tion fl20l)). then: 

max {VG(Xt)(I-Rf-D) > max{VG(Xt)(I-Rf ■D)+o(VG(Xt)) (38) 
Dev DgV 

D<Xt 



i.e, there is always an "almost" optimal feasible departure vector satisfying the 
condition Dt < Xt among the departure vectors that maximize the scalar product 

(VG(Xi)(/-i?f •!?). 
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Proof: We denote by I)VG'max ^ argmax£,g2?(VG(Xt)(/ - Kf ■ D), and 
by o'-p = min(i)7'~^™'''', Xt). Note that by construction: 

{VG{Xt){I - Rf ■ Df^) < {VG{Xt){I - Rf ■ Df^"^^^) (39) 

Furthermore note that by construction X^ = Xt — D). and fj^Gm&yi _ 
Di are orthogonal since the non null components of /j^Cmax _ j^\ ) _ 
max(i)7'^°''''' - Xu 0) coincide with the null of xf ^ = max(Xt - z)VGmax^ q) 
Thus according to (|20] |: 

(40) 
now expanding in Taylor series S/G{X) around point Xt we obtain 

VG(xf )) = VG{Xt) + i?W (X„ -D«) 

Since Dl is norm bounded, from (|34] i we can conclude that the remainder 

i?^^J.(Xt, -D^^^) is o{VG{Xt)) and thus: 

(VG(Xi)-(£)7^'"^"-Dj^))(/-i?)) = (VG(xf^)-p7^'^^"-i:)f^)(/-i?))+o(VG(Xt)) 

(41) 
from which the assertion follows recalling (|39T l and (|40] i. Indeed 



133 n^ 

(VG(Xt)-(^7^'^^^-D7^"^^^)(/-i?)) < (VG(Xt)-p7^'^^^-DfOU-^)) 



® (VG(xf))-(A^^"^'"-A^'^)(/-^)) + o(VG(Xi)) < o(VG(Xi)) 



Proof of Theorem |5} 

First, observe that since amvals ai^e assumed i.i.d. and service constraints ai^e 
assumed to be static, we have H = X. 

The idea of the proof is rather simple; G{X) can be interpreted as a Lyapunov 
function for the system. The stability of the network of queues follows from the 
fact that the drift conditions of Theorem |4]can be verified. 

We now evaluate the drift of G{Xt) for large values of Xt. By definition: 

A£ = E[G(Xt+i)-G(Xt) \Xt\= =E[G {Xt + At- Z^VGmax^^ _ ^^^ |^^] _^(^^) 
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and approximating G{Xt + At — dY'-^"^^^{I — R)) with its Taylor polynomial 
centered at Xt, we get: 

E [G {Xt + At- 2^7^— (/ - R)) \Xt] 
= G{Xt)+{'^G{Xt)-E [{At - d7«— (/ - i?))] )+i?g) {Xt, At - dY^^^^{I - R)) 



(42) 



where the remainder R^ {Xt, At - dY'^'^^''{I - R)), since both At and D^Gmax 
are a bounded nomi vectors, in light of (f36l) satisfies: 



lim 



R\^>{Xt,At-DfGm..^I-R)) 



II^.IKoo l|VG(Xt)|| 

thus: 

E [G {Xt + At- Z57G--(/ - R)) \Xt] 
= G{Xt) + {VG{Xt) ■ E [{At - DVGmax(j _ ^))]^ ^ o{\VG{Xt)\) (43) 

Where, {VG{Xt) ■ E[(^t - D7«"^a^(/ - R))]) = {VG{Xt) ■ A - Df^'^''''{I - 
R)) = {VG{Xt) ■ A) - {VG{Xt) ■ Z)VGmax(j _ ^)j sij^j,g ^,y assumption A(/ - 

R)^^ lies in the interior of V, an e' > can be found, such that also A{I — R)~^ + 
g/ nVGmax jigg ij^ p^ ^g obtain: 



(VG(Xt)-Z)7^'"^^(/-ii)) = ma^{'^G{Xt){I-Rf-D) = max{VG{Xn){I-Rf-D)+o{\\VG{Xt) 

D<Xt 



> {\7G{Xt){I-Ry •A(/-i?)^^+e'Z)7^°^^^) > {VG{Xt)-A)+e\\\7G{X, 



t)\\ 
(44) 

for a suitable e > 0. Observe that the second equation holds by virtue of Lemma|3] 
while the last holds since (^G'CXOlJ-^/gg^jjVGmax) = ^ > o by virtue of ^. 
In conclusion: 

E [G{Xt+i) - G{Xt) I Xt] < -e|lVG(Xi)|I + o{\\VG{Xt)\\) 

for large Xt, and therefore (IT2] | is satisfied since for any e" < e', a 6 > can be 
found such that: 

E [G{Xt+i) - G{Xt) I Xt] < -e"\\VG{Xt)\\ 

for ||Xt|| > b. 
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Furthemiore for any Xt : \\Xt\\ < b, G{Xt+i) - G{Xt) = G{Xt + At - 
j^VGmax^j _ ^-j-) _ Q(^Xt) is bounded. Indeed once again we recall vector || A* — 
^VGmaxjj_^)|| is bounded in norm. Let C be a bound for \\At-DY'^'^''''iI-R)\\ 
Now||Xi+i||| = \\Xt+At-DY'^'^''''{I-R)\\ < \\Xt\\ + \\At-DY'^'^''''{I-R)\\ < 
b + C 

Thus being G{X) continuous, and thus bounded over compact domains 
both from above and below: G{Xt^i) — G{Xt) < maxxj:||Xf||<b+c ^("^) ~ 
m.mx^;\\Xt\\<b G{X). The |VG(X)|-stability of the system of queues immedi- 
ately follows since lim||x||-s>oo VG(X) = oo (as result of LemmalTJ 

Proof of Theorem m 

The generalization to the case in which St is a non trivial Markov Chain can 
be carried out by sampling the process Yt in correspondence of the instants {t^} at 
which St,^ = So for some specific state Sq. From theory of DTMC (recalling that 
St has a finite number of states) immediately follows that {t^.} forms a sequence 
of non defective regeneration points for the system. Thus applying Corollary [T] 
we can prove the stability of the system of queues. To simplify the notation we 
assume traffic to be single-hop along our proof; however we wish to emphasize 
that the proof for the more general case goes exactly along the same lines and can 
easily recovered by replacing in the following derivation the departure vector at 

time t, DVGmax ^iti^ j)VGniay.^j _ jiy 

Again we select G{X) as Lyapunov function. Approximating G{X) with its 
second order Taylor s expansion, we get: 



E[G(X,,^J I FtJ 
= G{Xt,)+ < VG{Xt,y Y. (E[At-A^^'"'")] > +E 



Now, since all polynomial moments of vector J2t',^^ i^t — Df^'^^^) are, by 
construction, finite, (this because every vector At — Df^^^^^ is bounded in nomi 
and polynomial moments of zu = tfc+i — tk are finite,) from (|37| | we obtain that 

E Ul Ut,,Y!C\At - DVGmax\l ^ o(||VG(X,J||), i.e.. 
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E[G(X,,^J I y^J = G(X,J+(VG(X,J.E 






D 



VGrnaxN 



)+o(||VG(X, 



tft. 



Furthermore: 

(VG(XtJ.E 

= (VG(XiJ.E 



tfe+i— 1 



L» 



VGmaxN 



E (^* - A^ 



VGmax 





"tfc+i-i 


) = (VG(XiJ.E 


>:(A 

tk 


-| 


"tfe+i-i 


) + (VG(X,J.i? 


L(« 




tfe 



(46) 



rjVGniax , rjVGmax 



Dl 



VGmax \ 



)VGmax 



D 



VGmaxN 
t ) 



with: 



(VG(XtJ.E 



tfc+i— 1 

E(^* 



7-)VGmax\ 



) = (VG(XtJ.EM(A-A^ 



VGmax 



)) 



<-eE[zfc]||VG(XiJ|| (48) 



where the (first) equation follows from classical reward-renewal arguments i.e. 
while the following inequality is obtained exploiting the same arguments as in 
proof of Theorem m in particular observe that (VG(Xt) • A - D^'^Gmaxj-j - R)) = 
(VG(Xt) ■ A) - {VG{Xt)-Df'^'^'^''), and since by assumption A hes in the interior 
of V, an e' > can be found, such that also A + g'/)VGmax jj^^ ^^ -p. fj.Qjjj which, 
recalling Lemma|3]we have: (VG(Xj) • Z^VGmax^ ^ ^^^ ^^^ (VG(Xt) • D) = 



D<Xt 



maxB6i,(VG(X„) • D) + o{\\yG{Xt 



> (VG(X„) • A + e'Dl 



VGmax \ 



> 



(VG(X„) • A) + e\\VG{Xt) ||, where the last inequality follows from dH}- 



) 
(47) 
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The term: 






t—th 



tfc+1-1 



t—tu 



E 



tfe+i-i 



VGmax 7-)VGmax\ 



Y {"^{GiXt,) - G{Xt) + G{Xt)) ■ Dl^^^- - D] 



t=tk 



E 



tfc+i-i 



VGmax 7->VGmax\ 



Y (V(G(XiJ - G{Xt)) ■ I?,^^-- - R 



t=tu 



+E 



tfc+i-i 



Y {VG{Xt)-Dl^-^^--D] 



VGmax 7~) VGmax \ 



t=tk 



(V (G(XtJ - G{Xt)) ■ D^^^™^^ - D^Gmax^ ^ o(||VG(XtJ|) as an immediate 
consequence of (l34l ): in this regard, we recall that by hypothesis polynomial mo- 
ments E[1 l^tfc — -'^t 1 1''] are finite for any h; this again because {t^} is non defective 
sequence of stopping times and aixival vector is bounded. At last observe that the 
term (VG(Xt) • Df^^"^"^"^ - i:)^G'max^ ^,^j^ ^^^ ^^ positive, by construction. 
As a conclusion 

E[G(Xt,^J I XtJ -G(XtJ < -eE[zk] \\VG{Xt^)\\ + o{\\VG{Xt^)\) 

Therefore (IT4] |. is satisfied, since for any e" < eK[zk], a 6 > can be found such 
that 

E [G{Xt,^,) - G{Xt,) I Xt] < -e"\\VG{Xt,)\\ 

for ||Xt|| > b. 

At last, to show that (IT3]) is satisfied too, observe that for any Yt,, : \\Xtj^ \\ ^ b: 



(49) 



E 



tfe+i-i 



G{Xt,^,)]=E[G{Xt,+ Y (At-D^^^^^)) 



G(XiJ+(VG(X,J.E 



tfc+i-i 



E ^-A^ 



VGmax 



hn-1 



hoi 



tfe+i-i 






tfc+l-1 



E (^* - ^^ 



VGmax \ 
t ) 



( j; (^t - Z)7^--))'^o . {d'^'GXXt, + a Y (At-D 



VGmax\ 



tk 



can be easily shown to be bounded by G{Xt^ ) + vo for ^^ appropriate vq > 0, since 



i)E 



^tfc+i-l^^^_^ VGmax-) 



are bounded for every i, as before; ii) G{Xt^ 



{d'G){Xk 
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and its derivatives {d^G){Xk) are by assumption bounded over compact domains 
(in particular they are bounded over the domain X : ||X|| < 6) because G{X) € 
Coo[]^A/ ^ ^. jji) (d''oG){Xt^ + aYftT^'^i^t - D^Gmax-)^ j^ boundcd as 
before in Ught of (|22] |. The VG(X)-stabiIity of the system of queues immediately 
follows from Corollary [T] since lim||x||-i>oo VG(X) = oo (as result of Lemma[Tl). 

Proof of Corollary H 

Consider the Lyapunov function C{X) = ■j^G{X)'^'^^; denoting by Zt = 

At - Dt{I - R)y. 

G{Xt+i) = G{Xt + Zt) = G{Xt) + {G{Xt) ■ Zt) + B?(Xt. Zt) 

Now recalling (|32l ) and (|36] l. since by construction Zt is a bounded noim vector we 
can claim that: (G(Xj) • Zt) = o{G{Xt)), and R^{Xt,Zt) = o{\\VG{Xt))\\) as 
Xt ^ oo Thus: 



E [C{Xt+i) I ^t] ;^ E [{G{Xt + Z0)'+'] 

■ E \G{Xt) + {G{Xt) ■ Zt) + o((||VG(X)||' ^^^ 



h + l 
= hTl [{G{Xt)t^' + {h + l) n{G{Xt) ■ Zt)]{G{Xt)f + o{\\VG{X)\\ {G{Xt))\ 

Now considering Xt sufficiently large, such that G{Xt) is positive (we recall 
that G{X) -^ oo, for ||X|| — >^ oo and thus, it must be positive outside some 
compact set), and taking the average, from (|44] | we have: 

{G{Xt))''{VG{Xt) ■ At - Df'^^^^I - R)) < -e{G{Xt))''\\VG{Xt)\\ 

for some e > (see (l44l)). 

The ||X II '^-stability immediately follows, observing that i) by construction 

limiixiKoo ^^^^^^PP^ = oo; ii) for any Xt : ||Xt|| < b, T^[G{Xt + 
Yt)) — [G{Xt)) can be bounded by an appropriate constant vq (this be- 

cause Yt is bounded as well as G{X) is bounded (from above and below) over 
compact sets); 

The extension to the more general case can be canied out by observing that 
C{X) = j^G{X) is a strong potential provided that of G{X) is a strong potential 

by Corollary m Indeed denoting with Zt^ = Ett^'"^ {^t - Dt{I - R)), we have: 
E [CiXt,^,) I Xt] = CiXt,)+E[{VC{Xt,) ■ Zt,)]+E[RliXt„Zt,)] 
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withE[i?|(Xt„Zi,)] = o{C{Xt^))) andE[(/:(X,J ■ Z^,)] = o{C{Xt^)) by m 
and ( [37] ). since, by construction, all polynomial moments of Yt^. are finite. 
Now observing that VC{Xt^) = {G{X))^VG{X) we get: 

E [C{Xt,^,) I Xt] = C{Xt^)+n{G{Xt,f{VG{Xt,yZt^)]+o{\\{G{Xt^)f\\VG{Xt^: 
The assertion follows along the same lines as before. 

Proof of Corollary m 

The proof is rather straightforward. Under i.i.d. arrivals and static constraints 
(i.e. when % = X)sNe. can select the weak potential function G{X), as a Lyapunov 
function, then: 

From (l30l ) and (1361 ) we have: 

E [G{Xt + At- 1^7^— (/ - R) I Xt] - G{Xt) 



with {VG{Xt) ■ A - D^Gmax^j _ ^^^ < -e'||VG(Xt)|| for an appropriate 
e'>0. 

Now again from (|43]l, substituting D'^Gimp ^^ ^VGmax ^g j^^^g. 

E[G(Xt+At-i?^«'-P(/-i?)|Xi]-G(XO = (VG(Xi)-A-i?7^""P(/-i2))+o(||VG(Xi)||) 
and by assumption: 

(VG(Xi) • Z)7^'"^P(/ - R)) = {VG{Xt) ■ D^Gmax^^ _ ^)^ ^ o(|lVG(Xj)|l) 
Combining the two, we have: 

E [G {Xt + At- D^Gi'^P(/ - R)) I Xt] -G{Xt) = (VG(Xt)-A-Z)7^™P(/-i2))+o(|lVG(Xt 

= (WGiXt) ■ A - DVGmax(j _ ^)^ ^ oi\\WG{Xt)\\) 

Thus 

E [G (Xt + At- D^Gimp(^ _ ^)) 1^^] _ ^(^^) < _^„||^^^^^^ 

for some e" > 0. [VG(X)|-stability for the system of queues follows. The proof 
in the case in which G{X) is a strong potentials follows exactly along the same 
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lines. Furthermore adopting C{X) = jAr[[G{X)]^'^'^ as aLyapunov function and 
acting as before, the stability criteria can be strengthened. 

Proof of Theorem 111 

Proof: First, we recall that in this case the space state of the DTMC is Yt = 
[Xt, Dj^Gmcmj -^g ggjggj jj^g Lyapunov function 

C{Yt) = L(Xu d7«-^-) = L^(X^) + L^iXu D^'^^'^^, 

with 

jCi{Xt) = G{Xt) 

and 

C2iXt,DY^'"^'^) = {{VGiXt) ■ {D'^Gm^^ _ ^VGmcm^^j _ ^^^^Z? 

where /3 > 1 is given by ( |26l ); and we show that the drift condition of Theorem |4] 
is satisfied. 

First, observe that £2(^4,-07'^""'^™) is well defined because {VG{Xt) ■ 
(^VGmax _ jj^Gmcm^i^j - R)) > Q by construction. 

Taking the second order Taylor expansion centered in Xt for Ci{Xt+i) we get: 

CiiXt+i) = G{Xt+i) = G{Xt)+{VG{Xt)-At-Df''^'^{I-R) \ Yt)+Rl{X,At-Df^^^'"'{I-R)) 

and recalling that i?|,(X,y) = YHh{Xt + aY)Y'^ = 0{\\HG{Xt)\\) for any 
vector Y in light of (|T9l ). we obtain: 

K[/:i{Xt+i)-Ci{Xt) \Yt]=E [(VG(XO • At - Z)VGmem(j _ ^) I Yt)]+OmGiXt)\\) 

Now 



E 



= (VG(Xt) • A - (^7^?"^^^ _ £,VGmax _ ^VGmcm^^j _ ^^^ 
(VG(Xi) • (^VGrnax _ ^VGmcm^^j _ ^^^ ^ (VG(Xt) • A - Df^'^^''{I - R)) 



< iC2iXt,Df^^'^)y/(' -e{\\VG{Xt)\\) 

for an appropriate e > 0. Indeed by assumption A{I — R)^^ lies in the interior of 
V. 
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Thus: 



< {C2{Xt,D^^^'^)y/('-e{\\VG{Xt)\\) + 0{\\HG{Xt)\\) (50) 
Focusing instead on C2{Xt, Df^^^^^), we suppose for the moment Dt 7^ 

riVGmax. 



E 



VGmcm 



V rjVGmcm / rjVGmax 
TIN x^ T-iVGirLcm / /-iVGinax 7-iVGmcm / 7-iVGmax 

PrlDj"^^""""^ / Dj"^^"^^^ Xt, D^Gmem _^ ^VGmaxj 

I 17 /■ /' \^ 7-)VGmcm\ -ia 7->VGnicm 7->VGmax 7-1 VGmcm / T-iVGmax 

T>^) n VGmcm T-iVGmax 1 v n VGmcm / T-iVGmaxl 

< E[[{VGiXt+i) ■ P7+?-- - Df^^rmi - R))y\Xu 

^VGmcm ^ ^VGmax^ DYf^'^"'^ / 0^^^'^^''] (1 - 6) 

where the last inequality comes from the fact that by construction: 

= E [{{VGiXt+i) ■ [DY^f"^- - DY,f--^)iI - R))f] = 

while Pr{L>^^^™<'™ ^ ^^VGmax | x^, /^VGmcm ^ ^VGmax| <l_§ 

Now since our scheme guarantees that: {VG{Xt+i) ■ Df^^'''^{I - R)) > 
(VG(Xj+i) • D VGmcm (J - R)) wc Can write: 
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E 


[{vGiXt+^)iDYf,^^^-D7frnii-R))) 


V rjVGmcm / rjV&'max 


T-iVGmem / r-vVGinax 


<E 


'(^{VG{Xt+i)-{Df_^^^'^''^-Df^'^^'^){I-R))y 


Y rjVGmcm / r^VGmax 


(1-6) 


= E 


'(^{VG{Xt+i)-{Dff^"'^^-Df^"'^^+Df^'^^^-Df^'^^'^){I-R))y 


^t, 




rjVGmcm / r^VGmax 


(1-6) 




= E 


"((VG(X,+i).(A^+^--- 


d7^ 


^^")(/-i2)) 







(1-5) 



+ {VG{Xt+i) ■ {R 



VGmax _ ^ VGmem^ (I - R))) \ Xt, 



Dt + D\ 



VGniax 



(1-5) 



Expanding component-wise in Taylor series VG(X), we can write, similarly 
as before, VG(Xt+i) = VG(Xt + ^ - ZjVGmem^j _ ^)) ^ VG(Xt) + 
0( II FG(Xt) II) in light of (O, ([31 and of the fact that both At and D^Gmcm 
ai^e bounded. Furthemiore observe that by construction: (VG(Xj) • (£) VGmax _ 
^ VGmax) (J - i?)) > 0, therefore we obtain: 

E[/:2(^t+l, A+?'''"') I ^t with D^Gmcm _^ ^VGmax] 

< ((VG(Xt) • (ZjVGmax _ ^VGmcm)^^ _ ^^^^^ _ ^) ^ 0(||/?G(^i)l 



/:2(Xt, A^'''"''"')(i - <5) + o(/:2(Xi, z)7G-<=-)) 
< -<5£2(^t, A"""^"'") + o(/:2(^t, A"""""'")) (51) 



Thus: 

Er/' / \^ nVGmcmN i -v T-iVGmcm / T-iVGmaxi r I V nVGmcm 

\L-2\XtJr\^i^t+\ )\Xt,L)t f^ -LJt \ - l-2{Xt, L>t 



■>VGmom\ 



^VGmcm\ 



while for D.VGmcm ^ nVGmax. 



mrr tv nVGmcmN i v T-)VGmem n^Gmaxi 



E 



[{VG{Xt+MD7+r''''-Df'''^n{I-R)) 






E 



(VG(Xi).(Z)^ 



VGmax T-iVGmax 



t+1 



+ 0{\\HG{Xt 



< 0{\\HG{Xt)f)) (52) 



D 



lii-R)) 



^vcmax^^j _ ^)) is negative 



In light of the fact that (VG(Xt) • (D^^^™^^ - ^vcmax 

by construction, and (x)^ is monotone; indeed we recall that, by construction: 

(VG(Xt+i) • (A+^'^^^ - d7^"^^^)(/ - i?) > 0. 
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Combining together ^ and (EB or (EU, (we recall that £2(^4, A^'^""''"") = 

E[/:(x,+i,z57+^-''-)|yt] 

= -ei\\\/GiXt)\\) - 6C2iXt,Df''^'-'))+ (53) 



in light of (I261 ). Thus for a sufficiently large 6 > 0, we can claim that: 

E[/:(Xi+i, A+'l'"'"') I Yt]-C{Xt,Df''-''-^) < -e'{\\VG{Xt)\\) 

for any Yt], such that ||Xt|| > b and for any e' < e (in this regard, we recall that /? 
is selected in such a way to guarantee that ||i/G(Xi)||^ = o(||VG(Xj)||)). 

\VG(X) j-stability for the system of queues follows, since for any Yt : \\Xt\\ < 
b, £(yt_|_i) — C{Yt) is bounded, as immediate consequence of the fact that ||^t+i — 
Xt\\ is bounded. 

The stability criteria can be strengthened. For any /i € N, we can 
prove that the system of queues is HX'^H -stable under any admissible arrival 
vector, by selecting the Lyapunov function C'(Yt) = C'{Xt,Df'-''^'"^) = 
^(£(X,,A^^-<=-))"+^ = T^,{C,{Xt) + C2iXt,DfG-^--^))''^' = 
j^{G{Xt) + ((VG(Xj) • (DVGmax _ ^VGmem)(j _ ^)))'3)'^+\ The deriva- 
tion proceeds along the same lines as the proof of Corollary |2l essentially showing 

thati?[£'(yi+i) I Yt] -c'{Yt) « (£(Xt,D7«— ))''(i<;[/:(Xi+i,A^Gmem) j 

Yt] - CiXt.D^Gmem-^'^ < -e{\\VG{Xt)\\){C{Xt, D^^"^-"^))^ , for any Yt : 
\\Xt\\ > b with a sufficiently large b > 0, and for a sufficiently small e > 0. 

Proof of Theorem m 

This proof combines arguments already applied in the proofs of Theorem [6] and 
Theorem |7] First, we observe that the state of the DTMC representing the system 
evolution is given by vector: Yt = [Xt, St, St^], where St € S"^ x 5^ represents 
the dynamic of exogenous aixivals, and sei^vice constraints, while S^ provides the 
additional information about the memory state of the scheduling algorithm; such 
information correspond to the set of departure vectors Dt^ {S^ ) , memorized by the 
scheduling algorithm. We recall that, by construction, Dt\S^) is the departure 
vector employed at the last occurrence of state S^ before t. 

We select a Lyapunov function with a similar structure as the one used in Theo- 
rem|7] however this time, things are made slightly more difficult by the fact that the 
memory of the scheme is significantly larger. Furthermore the stability properties 
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of Yt are derived from those of the DTMC y^^ obtained through the sub-samphng 
of Yt, at instants {t^} in which the DTMC St^ = So, for a particular state ^o 
(Corollary [T}. Again we can claim that {tk} forms a sequence of non defective 
regeneration points for the system since St is finite state and ergodic. 
In more detail the selected Lyapunov function is: 

CiYt) = C{Xt,Sf') = Ci{Xt)+C2{Xt,Sf'), 

with: 

Ci{Xt) = G{Xt) 
and 

C^iXt, Si') = Y,ns {{VG{Xt) ■ P7«--(5) - Df'iSW - R))f , 
ses 

where, once again, we recall that Dl^{S) represents the memorized departure vec- 
tor that corresponds to state S (i.e., to the component S^ of 5 associated with 
service constraints), /3 > 1 is specified by (l26l ) and vr^ is the steady state probabil- 
ity of the DTMC St governing arrivals and dynamic constraint conditions. 

In the remainder of the proof to simplify the notation we omit the depen- 
dency of the departure vector on constraints conditions, writing /j^G'^cm instead 
of Dt^^^'^'^{Sl^), whenever this can be done without causing confusion. 

Taking the second order Taylor expansion centered in Xt^ for G{Xtf.j_-^) we 
get: 



W.[Ci{Xt,^,)-Ci{Xt,)\Yt] 



E 



Cfc+l 



-1 



Y, (VG(Xt) • At - l)7^---(/ - R)) I Yt, 



t—tu 



+ 0{\\HG{Xt 



with: 

E 

= E 



"^fc+i" 



(VG(X,,). Y. At-D'?^^^^{I-R)>\Yt, 

t=tk 

(VG(XiJ. Y ^t-{D. 
t=tk 

tfe+i-i 
(VG(XtJ- Y (^: 



VGmax rjVGmax _, T^VGrncm 



)ii-R))\Yt 



tk 



E 



VGmax _ ^VGmcm^^j _ ^^^ j y^ 



tk 



t—tk 



+ E 



(VG(X,,) • ^ A, - Z)VGmax(^ _ ^)^ I Y^^ 
t=tk 
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Now, E 

-eE[zfc](| 
While: 



ESr (VG(XO . At - DVGmax(j _ ^)^ | 

WG{Xt) II), from (|47]l and (I48]l in the proof of Theorem |6] 



y. 



tfc 



< 



E 



E [(VG(XtJ • Y, (I^VGmax _ ^VGmem)(j _ ^)^ | y^^ 
l\7f<(V V \ ^ / rjVGmax T^VGmcm , nVGmax/' c£'^ T^M f qD\ nVGmax/' c'£'^ i nM f qD 



<E 



+A^^--(S/^)-Di^^(5,^)-A^^^--(5,^)+Di^(5f ))(/-i?)) I 1 






where we recall that 



^vGmax^^i.) = arg max(VG(X, J • 0(1 - R)) 

and D^^{SP) is the departure vector memorized by the scheduling at time 
tfc, which corresponds to state Sj^. Observe, indeed, that E (VG(Xt^, ) • 

EtT'iDf'^'^^^ - Dl^-^-^{Sf)){I - R)) I Yt,] 

of Dl^'-^^{Sf) and E [(VG(X,,) • YltT' D^{SP 



< from the definition 

VGincmN 



^Vt.mcm)(j_^)^ 



Oil 



= E 

HciXt) 
Now: 



Ettr'(VG(X,) • D^^iSP) - Df^-^^-^){I - R)) I Yt 
), with {VG{Xt) ■ (D^MSP) - d7^°^^>^)(/ -R)) <0 from 



+ 



E 



i^fe+i- 



(VG(x,,) . j; (A!'''"^^(5t) - Df^iStW - R)) I ^t 



tfe 



t=tfe 



E[^.] 5; ^5(VG(X,J . (Dl^'^^^iS) - DfliSmi - R)) 

= E[z,]C2{Xt„St^))'/^ 



Thus: 



E[A(Xi,^J-A(Xt,) I Yt,] < E[zk]{C2{Xt,,Sl^)y/^-e\\{VGiXt.km+Oi\\HG{Xt 

(54) 
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Focusing instead on £2(^tfe, S'j/), first observe that Vt G [tk,tk+i], stopping 
time for the Markov Chain SP, we have: 



E 



{VG{Xt,^MDZ^r''iS)-Df^^,{S)){I-R))) I Df^iS) = Of^^^^iS) 
(VG(Xi,^J • (A!S""(5) - Df^-^^^SW - R)) 



E 



M 



+ (VG(X,,^J.(A^^'"-(5)- A^,(5))U-^)) I D^iS) = D 



iA// 



)VGmax 



E 



(S) 

M 



/3 



/31 



<E 



(VG(X,,^ J • (A:™(5) - Z). 



-iVGmax 



(smi-R)) 



(55) 



because by construction (VG(Xj^_^^ ) • (Z)f (5) - Df^^^ {S)){I - R) < and (x)^ 
is monotone (observe that by construction the arguments of the power operator is 
positive). Now: 



E 



^((VG(X,,^J 


■ (Dl^^^^iS) - Df^'^^^{S)){I - i?))) " 




= E 


({VGiXt) ■ (A'lfr^'' - Df^'^^''){I - R)) 


+ 0{\\Hg{ 


mi)y] 


<0{\\HG{Xt)\\y = o{\\HG{Xt^^, 



(56) 



where first equation is a direct consequence of (I35> and the fact that polynomial 
moments of tf^^i — t are finite, the following inequality is a consequence of the fact 
that {VG{Xt) ■ (L»t^'j^^"^^^(5) - DY'^'"''''{S)){I - R)) is negative by constmction, 
and (x)^ is monotone Thus combining (1551 ) and (1561 ) Vi € [tk, ifc+i]. stopping time 
for the Markov Chain 5^^, we get: 



E 



((VG(X,,^J.(A!?r'(5)-Af+,(5))U-^)))T I Df'iS) = Df^-^'^^S)] = 0[\\HG{Xt\\) 



(57) 
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From which we can conclude that: 



E 



[{VG{X,^^MDl^r^''{S)-Df^JS)){I-R))y I Df^iS) + D\ 



VGmax 



(S) 



E 



E 



/3 






/3 






VGmax 



(S), 



E 



({WG{Xt,^MDZ^riS)-DtUsmi-R)) 



OmoiXtk)]]) (58) 



where tts denotes the probabihty state S is visited in (tfci^fc+i] (i-e-» be- 
tween to following visits to state Sq). Indeed observe that by construction 
Pr {Df (5) / DfG^-^iS) Vt G (tfc,tfc+i] I Al'(5) y^ DY^Gm..(^s)} < I-Stts 
since D^ (S) = jj^GraioL ^j^j^ ^ probability greater than 5 provided that 
state S has been visited at time t = (i/c,i/c+i], and we can apply ( [57] ) to 



E 



((vG(x,,^j • (A!^r'(5) 



D^'UiSW -R))) I 3t G (tfc,tfc+i] : 



At last 



E 



<E 



(VG(Xt,^J • (Dl^^^^iS) - Df^^^{S)){I - R)) 

< E [((VG(X,,^J . (A!S""(5) - Df^iSW - R))y 
((VG(X,J . (dZ^^^^S) - Df^{S)){I - R)) + 0{\\Hg{X, 



(59) 



where the first inequality derive from (US) while the following inequality can be 
obtained expanding in Taylor series G{Xt^_^^ ) around Xt^ and observing that by 
construction VG(XtJ • (Z)^^<^'"^^(5) - DZ'^^^''{S)){I - R) < 0. 
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Thus combining (1581 ) and ( 1591 ) we get: 



E 
<E 



((VG(x,,^j.(A!?r'(5)-Al,(5))U-^)))^ I <(^) + Dl^'^^^s) 



((VG(X,J.(A^^--(5)-A*^(S))(/-i?)(l-5vr5))J +o((VG(X,J.(Z),^^--(5)-Z?4^(5))(/-i? 



Now multiplying for tts and summing over all the states and recalling (|60] l and (|57| i 

J -(AT 



and recalling that VG(Xt,) • (Z);^^°^'^=^(5) - D^l{S)){I - R)) = if A,. (5) 



= ^^TTsE [((VG(X,,^J • (A!Jr'(5) - Df^SW - R))y I Xt„S,f ■ 
s 

<Y,^s{l - Sns) [((VG(X,J • (DZ^^^^S) - Df^{S))iI - R)) 

s 

^7Ts{{VGiXt,)-iDZ^^^^S)-Dfl{S)){I-R))f)+oi\\HG{Xaf) 



< E[C2iXt„Sf^)]{l - 6mm7rs7rs)) + o{/:2(.Xt„S^^)) + o{\\HG{Xt,)f) 



(61) 



where observe that min^ tts'^s > since S^ is a finite state ergodic Markov 
Chain. 

At last, by combining ( [6T| ) and ( [54] ). we can easily show that drift condition 
(IT4l ) is satisfied. 

|VG(X)|-stability for the system of queues easily follows, since (|T3] ) can be 
easily derived (as for the previous Theorems) from the following three facts: i) 
C{Y) and is indefinitely continuously derivable, and thus bounded (along with its 
derivatives) over compact domains; ii) at any instant t both arrival and departure 
vectors are bounded; iii) K[zl] are bounded for any i > 0. 

We further notice that the stability criteria can be strengthened. For any h € 
N, we can prove that the system of queues is ||X'^|| -stable under any admissible 
arrival vector, by selecting the Lyapunov function C'{Yt) = C'{Xt,DY^^'^^) = 
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